Skip to content

feat(search): add Serply as a web search provider - #178

Open
googio wants to merge 1 commit into
deer-flow:mainfrom
googio:feat/serply-search-provider
Open

googio wants to merge 1 commit into
deer-flow:mainfrom
googio:feat/serply-search-provider

Conversation

@googio

@googio googio commented Sep 13, 2026

Copy link
Copy Markdown

Summary

Adds Serply as a search provider for the built-in web_search tool, selectable in Settings > Search. Serply returns Google SERP results over a plain REST endpoint (GET https://api.serply.io/v1/search/, key in X-Api-Key), which is the one thing the current lineup doesn't cover: none of Firecrawl, Tavily, Brave, Exa, AnySearch, or Zhihu returns actual Google rankings. The key is required, and there's no anonymous tier. Docs: https://serply.io/docs

Nothing changes for existing users. The default provider is still Firecrawl, and the Serply path is only reached when a user picks it in settings.

The wiring follows #163 file for file: SearchProviderId and SearchSettings, the provider factory, settings persistence, and the Settings page entry with its "Get API key" link. SerplySearchProvider implements the same SearchProvider interface as its neighbors, and web_fetch delegates to Firecrawl the way BraveSearchProvider does, since Serply has no single-page extraction endpoint.

Three implementation notes, since they're the points raised in the review on #151:

The limit is enforced on both sides. One Serply request reads a single result page and a page carries at most ten organic results, so num goes through the existing _clampedLimit() (1..10) instead of being truncated server-side, and the response is sliced to the caller's limit as well. A SERP crowded with non-organic blocks can come back with fewer, so the count is a ceiling rather than a guarantee.

Two headers go out, Accept and X-Api-Key. No client IP, nothing forwarded.

The body is parsed defensively. Serply reports errors as JSON ({"detail": "..."}) but sits behind a CDN that can answer with an HTML page instead, so parsing blind would bury a 502 under a syntax error. The text is parsed inside a try; a failed parse leaves the status visible, and a non-JSON 200 is reported rather than treated as an empty result set.

LangGraph generator parity

Per AGENTS.md the generated Python moves with the TypeScript:

  • _serply_search() in web_search.py, mirroring _brave_search() with the same clamp, slice, and error handling.
  • SERPLY_API_KEY in the generated .env and .env.example.
  • _hasLiteralSecret() now counts serplyApiKey. Without that, a project configured with a literal Serply key and a $VAR model key got no .env at all and silently lost the key.
  • built-in-sources.generated.ts regenerated with bun scripts/gen-langgraph-tools.ts.

AGENTS.md asks for the generated Python to be executed rather than only asserted against as a string. It was, against a stubbed requests, covering the same fifteen behaviors the TypeScript tests cover. Both runtimes were then run once against the live API with the same query and returned the same three results.

Test plan

  • 6 mock-fetch tests in packages/runtime/tests/tools/built-in/web.test.ts: endpoint and auth header with the normalized result shape, the page cap plus local slice, a missing key, an error detail surfaced verbatim, a non-JSON body, and web_fetch delegation to Firecrawl.
  • 3 env-block regression tests in packages/core/tests/generator/langgraph/templates.test.ts.
  • bun test goes from 1013 to 1023 passing, no new failures. Five tests fail on my machine both before and at this commit, from a missing rg binary and two environment-specific cases, none of them in search.
  • bun run check:changed is clean for lint and typecheck.

One thing left out on purpose

docs/settings.md and docs/settings.zh-CN.md list only Firecrawl and Tavily in their provider table. Brave, Exa, AnySearch, and Zhihu are all missing there already, and the section carries a screenshot that would need retaking, so a lone Serply row would sit oddly. Happy to send a separate PR bringing the whole table and the screenshot up to date if that's useful.

On #151

#151 adds SearXNG across nearly the same files, so whichever lands first leaves the other with conflicts. I'm glad to rebase on top of it or wait, whichever you prefer.

Disclosure: I work with Serply. Happy to adjust scope, naming, or drop this entirely if it isn't a direction you want for the project.

- Extend SearchSettings with serplyApiKey and wire "serply" into the
  provider factory, the search settings persistence, and the
  Settings > Search page (Get API key link plus the required-key
  semantics; Serply has no anonymous tier).
- Add SerplySearchProvider, a plain REST client for Google SERP results.
  num is clamped to a single result page (10) and the response is sliced
  to the caller's limit, and the body is parsed defensively so a CDN HTML
  error page surfaces the HTTP status instead of a JSON syntax error.
- web_fetch on Serply delegates to Firecrawl for safe page extraction,
  matching the Brave provider; Serply exposes no single-page extraction
  endpoint.
- Mirror the provider in the LangGraph generator so a generated project
  runs the same backend: _serply_search in the embedded web_search.py,
  SERPLY_API_KEY in the generated .env/.env.example, and the literal-key
  check that decides whether .env is written at all.
- Add en/zh labels for the provider and its key.
- Cover the new paths with mock-fetch tests (endpoint and auth header,
  result normalization, the page cap, a missing key, an error detail, a
  non-JSON body, and web_fetch delegation) plus generator env-block
  regression tests.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant